A Problem-Solving Method for Plan Evaluation and Critiquing

As intelligent systems become larger and work in increasingly more complex and knowledge intensive environments, it becomes impractical to develop knowledge bases from scratch. A practical alternative is to develop knowledge bases out of reusable components, either ontologies or domain-independent problem solving methods. This paper describes a reusable knowledge base of general principles about plan evaluation and critiquing. It includes ontologies to represent plans and critiques, an ontology of evaluation criteria, and problem-solving knowledge about how to evaluate plans with respect to those criteria. Currently, the evaluation criteria analyze various aspects of the plan structure and its use of resources. We performed a study of three evaluation and critiquing task domains, and found that our framework covers approximately 75 percent of the critiquing criteria related to plan structure and resource use (and over 90 percent within the framework's declared scope). Our framework also enables a systematic approach to the development of plan evaluation tools by following the organization of the criteria in the method.